Learning and controlling the source-filter representation of speech with a variational autoencoder

Understanding and controlling latent representations in deep generative models is a challenging yet important problem for analyzing, transforming generating various types of data. In speech processing, inspiring from the anatomical mechanisms phonation, source-filter model considers that signals are produced few independent physically meaningful continuous factors, among which fundamental frequency $f_0$ formants primary importance. this work, we start variational autoencoder (VAE) trained an unsupervised manner on large dataset unlabeled natural signals, show production naturally arises as orthogonal subspaces VAE space. Using only seconds labeled generated with artificial synthesizer, propose method to identify encoding first three formant frequencies, these orthogonal, based orthogonality, develop accurately independently control factors within subspaces. Without requiring additional information such text or human-labeled data, results spectrograms conditioned applied transformation signals. Finally, also robust estimation exploits projection signal onto learned subspace associated $f_0$.